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2.  Original  Objectives 

Initially,  the  aim  of  this  project  was  to  analyze  in  more  detail  the  following  aspects  of  the 
(a)  cognitive  mapping  system  and  (b)  the  action  system  of  models  that  perfom  spatial  cognitive 
tasks. 

Cognitive  mapping  system: 

1.  Place  discrimination. 

a.  The  cognitive  mapping  system  will  generate  place  representations  whenever  an 
unfamiliar  spatial  location  is  visited. 

b.  Place  discrimination  will  include  a  more  detailed  description  of  the  processes 
that  relate  landmark  (object)  recognition,  the  computation  of  visual  angles  of  the  recognized 
landmarks,  the  perception  of  directional  (local)  views  of  the  environment,  and,  finally,  the 
integration  of  different  local  views  into  place  representations. 

c.  Place  discrimination  will  be  accomplished  using  extramaze  or  intramaze 

landmarks. 

2.  Place  interconnection. 

a.  The  cognitive  map  will  represent  not  only  adjacency,  but  also  accessibility.  This 
will  allow  the  cognitive  map  will  to  represent  different  environmental  constraints,  such  as  opaque 
and  tranparent  walls,  obstacles,  etc. 

b.  The  cognitive  map  will  represent  not  only  accesibility  but  also  distance  and 
direction  between  places. 

3.  Reward  location: 

a.  The  cognitive  map  represents  the  location  of  food  in  the  maze. 


Action  system: 

1.  Decision  making. 

a.  Instead  of  a  fixed  decision  time,  decision  making  rules  will  reflect  a  variable 
decision  time  as  a  function  of  the  knowledge  of  the  environment. 

2.  Response-selection  rules. 

a.  Response-selection  rules  will  replace  VTE  behavior  and  stimulus-approach  rules 
when  the  maze  is  well  learned. 

b.  Response-selection  rules  will  be  inhibited  when  the  fast-time  prediction 
associated  with  a  given  response  changes  in  response  to  changes  in  the  structure  of  the  maze. 

3.  Environmental  exploration. 

a.  Exploration  of  the  environment  during  global  cognitive  mapping  will  be  a 
function  of  the  knowledge  of  the  environment  (curiosity). 

4.  Navigational  dynamics.  The  model  will  describe  acceleration,  velocity,  and  position 
of  the  system  as  a  function  of  time  during  spatial  navigation. 


3.  Status  of  the  effort 

The  unexpected  early  termination  of  the  contract  generated  havoc  among  the  personnel 
involved  in  the  project.  At  the  beginning  of  the  project,  we  had  a  difficult  time  finding  a 
postdoctoral  student  qualified  for  a  job  that  requires  knowledge  of  computer  science,  animal 
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behavior,  and  neurophysiology.  Finally,  we  hired  Dr.  Peter  Williams  in  January  of  1995.  Initially, 
I  trained  Dr.  Williams  in  the  intricacies  of  modelling  animal  behavior,  and  we  wrote  some 
programs.  Almost  as  soon  as  he  started,  we  received  the  notification  that  the  grant  would  be 
terminated  in  June,  and  he  started  searching  for  another  job. 

We  managed,  however,  to  complete  several  of  the  original  objectives  of  the  project  and 
even  extended  the  enterprise  to  include  the  building  of  real  robots  that  perform  spatial  learning 
and  cognitive  manning. 


4.  Accomplishments/New  Findings 

1.  Place  discrimination. 

The  cognitive  mapping  system  generates  place  representations  whenever  an  unfamiliar 
spatial  location  is  visited. 

2.  Response-selection  rules. 

a.  Response-selection  rules  will  replace  VTE  behavior  and  stimulus-approach  rules  when 
the  maze  is  well  learned.  We  developed  a  sofisticated  model  capable  of  generating  alternative 
responses  in  the  presence  of  different  cues  and  of  modulating  the  intensity  of  these  responses 
according  to  the  location  of  the  animal  in  the  maze.  For  instance,  given  three  alternative  paths 
(A,  B,  and  C)  and  three  alternative  approach  responses,  the  model  is  capable  of  learning  to 
approach  A  in  spatial  location  X,  B  in  spatial  location  Y,  and  C  in  spatial  location  Z.  Or  given 
two  different  motivations  (thirst  and  hunger),  the  model  is  capable  of  learning  to  approach  a 
water  source  when  thristy  and  the  food  source  when  hungry  performing  the  adequate  response 
in  each  case. 

3.  Searching  Behavior 

One  of  the  most  important  applications  of  spatial  learning  and  cognitive  mapping  in 
searching  behavior.  In  some  cases  the  objective  of  a  search  behavior  is  to  encounter  and  remove 
of  all  targets  (Figure  1,  Left  Panel),  in  other  cases  the  objective  is  to  encounter  and  remove  a 
group  of  selected  targets  to  open  a  path  between  Start  and  Goal  points  (Figure  1,  Right  Panel). 

In  both  cases,  several  factors  are  important  in  guiding  searching  behavior: 

1  ■  Item  distribution:  The  targets  can  distributed  (a)  randomly  (Figure  2,  Panel  A), 
(b)  uniformly  (Figure  2,  Panel  B),  (c)  in  concentrated  patches  (Figure  2,  Panel  C),  (d)  in 
dispersed  patched  (Figure  2,  Panel  D),  (e)  with  some  spatial  periodical  probability. 

2.  Search  strategies:  Alternative  searching  strategies  include  (a)  random  walks 
(Figure  3,  Panel  A)  ,  (b)  straight  line  walks,  (c)  moving  around  the  perimeter  and  inwards  to  a 
central  point  in  the  search  area  (Figure  3,  Pcinel  B),  (d)  moving  slowly  and  turning  when  the  hit 
rate  is  high  in  order  to  stay  within  the  boundaries  of  the  patch  (Figure  3,  Panel  C). 

3.  Percepftial  learning:  The  agent's  ability  to  detect  the  targets  improves  as  a 
function  of  recent  encounters  with  similar  items. 

4.  Social  cooperation:  Search  behaviors  might  benefit  from  social  cooperation 
because  a  group  of  agents  searching  for  a  concentrated  patch  is  more  likely  to  encounter  the 
patch  than  individual  agents.  Once  the  patch  is  found,  all  agents  assemble  at  the  found  patch  (See 
Figure  3,  Panel  D).  We  have  designed  a  model  of  animal  communication  that  will  be  applied  to 
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this  task. 


Spatial  Ipamitip;  Rate  of  success  might  be  improved  by  learning  about  the 
spatial  location  of  the  patch.  We  have  designed  a  model  of  spatial  learning  that  defines  the 
location  of  the  targets.  Figure  4  (Top  panel)  shows  that  the  location  of  the  goal  (G)  is  learned 
using  only  the  visual  angles  to  the  square  and  round  landmarks  and,  therefore,  generalization 
surface  A  presents  minima  at  G  and  G'.  After  training,  when  the  rat  starts  at  S  it  reaches  G, 
when  the  rat  starts  at  S'  it  reaches  the  miror  image  of  G,  G'.  Figure  4  (Bottom  Panel)  shows  that 
the  location  of  the  goal  (G)  is  learned  using  both  the  visual  angles  to  the  square  and  round 
landmarks  and  the  visual  angles  between  the  landmarks  and,  therefore,  generalization  surface  A 
presents  only  one  minimun  at  G.  After  training,  when  the  rat  starts  either  at  S  or  S  it  always 
reaches  G. 

3.1.  Formal  Models  of  Searching  Behavior 

We  started  studing  formal  models  of  searching  behavior.  The  models  analyze  at  a 
theoretical  level  strategies  that  optimize  the  search  and  maximize  the  rate  of  encountering  the 
target.  In  general,  models  are  defined  by  (a)  the  different  tasks  the  agent  can  do  simultaneously 
(searching  or  removing  targets),  (b)  the  probability  of  encountering  a  target  in  a  patch,  (c)  the 
agents'  ability  to  detect  targets  with  increasing  search  velocity  and  increasing  distance  to  the 
target. 

Formal  models  of  searching  behavior  are  important  to  (a)  analyze  optimal  behavior  under 
different  constraints  (limited  energy  or  limited  time  to  encounter  a  given  percentage  of  targets) 
and  (b)  evaluate  the  performance  of  alternative  network  designs. 

3.2.  Neural  network  models  of  search  behavior 

We  are  simulating  models  that  (a)  use  alternative  searching  strategies,  (b)  have  different 
detection  capabilities,  (c)  include  or  exclude  spatial  learning,  and  (d)  include  or  exclude  social 
communication,  in  (e)  environments  with  different  target  distributions. 

We  have  started  by  designing  different  neural  networks  that  control  search  behavior. 
These  models  search  a  simulated  environment  in  which  the  targets  are  presented  in  a  patch.  A 
measure  of  the  efficiency  of  the  network  is  the  rate  of  encounters  with  the  targets  (number  of 
targets  detected  per  unit  of  time). 

So  far,  we  have  analyzed  several  problems: 

1.  We  have  compared  different  strategies  against  the  simplest  possible  tactic,  i.e.,  random 
walk.  A  very  simple  strategy  that  dramatically  improves  the  efficiency  of  the  search  (in  terms 
of  time)  is  to  make  search  speed  a  decreasing  function  of  the  rate  of  encounters  (See  Figure  5). 

2.  We  have  designed  a  network  capable  of  describing  patch  density  at  different  points  in 
space  in  a  manner  that  is  independent  of  the  velocity  of  the  search.  The  network  is  able  to  find 
the  boundaries  of  the  patch  and  to  stay  within  those  boundaries. 


3.3.  Building  and  testing  autonomous  robots 

Based  on  computer  simulated  results,  we  are  building  small  autonomous  robots  controlled 
by  the  same  neural  networks  used  in  the  simulations.  The  robots  are  able  to  search  for  and 
approach  a  source  of  light  according  to  the  internal  state  of  the  agent.  We  have  now  one  small. 
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inexpensive  robots  (see  Figure  6)  that  we  will  use  to  test  searching  behavior  in  an  experimental 
arena.  The  floor  of  the  arena  contains  a  set  of  targets  (light  bulbs)  that  can  be  distributed  in 
alternative  ways  (see  Figure  2)  and  can  be  "removed"  (turn  off)  by  the  robot. 

5.  Personnel  Supported 

1.  Nestor  Schmajuk 

2.  Greg  Gale  (Graduate  Student) 

3.  Columbia 

4.  Columbia 
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contacted  and  visited  Duke  in  April  1996  to  exchange  information  about  spatial  learning  and 
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rnpnitivp,  mapping  in  animals  and  robots.  We  wrote  a  joint  proposal  for  MURI. 


C.  Knowledge  resulting  from  our  efforts  is  used. 
See  B  above. 


8.  Inventions 

None. 

9.  Honors/Awards 

Duke  International  Award 
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SPATIAL  LEARNING  AND  COGNITIVE  MAPPING; 
A  NEURAL  NETWORK  APPROACH 


Grant  No  F4920-94-1-0238 

Nestor  Schmajuk,  P.I. 

Columbia  College  of  Chicago 
600  S.  Michigan  Avenue 
Chicago,  IL  60605 

Period  Covered  1  Jun  94-  31  Dec  95 


Abstract 

The  aim  of  this  project  was  to  analyze  in  more  detail  the  following  aspects  of  the  (a) 
cognitive  mapping  system  and  (b)  the  action  system  of  computer  models  that  perfom  spatial 
cognitive  tasks.  Despite  the  early  termination  of  the  contract,  we  completed  several  of  the  original 
objectives  and  built  small  robots  that  perform  simple  spatial  tasks. 


Figure  1.  Left  Panel;  Spatial  distribution  of  targets  in  the  environment  and  movements  within  and 
between  patches  when  all  targets  should  be  removed.  Right  Panel;  Spatial  distribution  of  targets  in 
the  environement  and  desired  movement  when  the  targets  blocking  the  path  between  Start  to  Goal 
(lined  patches)  should  be  removed. 
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Figure  2.  Spatial  distribution  of  targets.  (A)  Random,  (B)  Uniform,  (C)  Concentrated  patches, 
(d)  Dispersed  patches. 


Figure  3.  Search  strategies.  (A)  Random  walk,  (B)  Contracting  square,  (C)  Patch-edge 
recognition,  and  (D)  Social  cooperation  between  Agents  1  and  2  when  Agent  2  finds  the  target. 


Figure  4.  Place  learning.  Top  panel:  The  location  of  the  goal  (G)  is  learned  using  only  the 
visual  angles  to  the  square  and  round  landmarks  and,  therefore,  generalization  surface  A  presents 
minima  at  G  and  G'.  After  training,  when  the  rat  starts  at  S  it  reaches  G,  when  the  rat  starts  at 
S'  it  reaches  the  miror  image  of  G,  G’.  Bottom  Panel:  The  location  of  ±e  goal  (G)  is  learned 
using  both  the  visual  angles  to  the  square  and  round  landmarks  and  the  visual  angles  between 
the  landmarks  and,  therefore,  generalization  surface  A  presents  only  one  minimun  at  G.  After 
training,  when  the  rat  starts  either  at  S  or  S’  it  always  reaches  G. 
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Figure  5.  Search  Strategies.  Time  to  encounter  95%  of  the  targets  using  a  random  walk  with 
(A)  constant  speed,  (B)  decreasing  speed  with  successive  encounters  and  initial  speed  of  20 
pixels/time  unit,  and  (C)  decreasing  speed  with  successive  encounters  and  initial  speed  of  10 
pixels/time  unit. 
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Figure  6.  Experimental  Robot.  Diagram  of  the  robot  designed  to  perform  searching  behavior. 
Sensor  1  is  an  ultrasonic  detector  that  senses  the  presence  of  walls  in  the  direction  of  movement  of 
the  robot.  Sensor  2  is  a  photoreceptor  that  senses  the  presence  of  active  targets. 


